Implementation and Optimization of Differentiable Neural Computers

We implemented and optimized Differentiable Neural Computers (DNCs) as described in the Oct. 2016 DNC paper [1] on the bAbI dataset [25] and on copy tasks that were described in the Neural Turning Machine paper [12]. This paper will give the reader a better understanding of this new and promising architecture through the documentation of the approach in our DNC implementation and our experience of the challenges of optimizing DNCs. Given how recently the DNC paper has come out, other than the original paper, there were no such explanation, implementation and experimentation of the same level of detail as this project has produced, which is why this project will be useful for others who want to experiment with DNCs since this project successfully trained a high performing DNC on the copy task while our DNC performance on the bAbI dataset was better than or equal to our LSTM baseline.